Statistical Learnability of Generalized Additive Models based on Total Variation Regularization

where x ∈ R denotes a sample and xj ∈ R denotes the j-th explanatory variable for each j ∈ [1, p] , {j ∈ N|1 ≤ j ≤ p}. This was first proposed by Hastie and Tibshirani (1987) and is known as a generalized additive model (GAM). In this paper, we call fj(·) a weight function and f(·) a GAM predictor. This not only includes linear predictors but also captures nonlinear relationships between explanatory variables and the targeted values. Although complex interactions or dependencies among explanatory values are not expressed, GAM predictors are expected to exhibit higher predictive performance when properly learned from a sufficiently large amount of data, at least in comparison with simple linear models. There has already been substantial work on data mining and statistics using GAMs (Guisan et al., 2002; Wood, 2006). We first introduce the total variation (TV) of a function as a measure of complexity of functions in Lc(R)-space. Here, L 1 c(R) denotes a space of functions with compact support in L 1-space on R. Secondly, we introduce the sum of TV among all weight functions as a natural measure of complexity for GAM predictors: